Calibrating Confidence Coefficients

Two approaches for dealing with the problem of poor coverage probabilities of certain standard confidence intervals are proposed. The first is a recommendation that the actual coverage be estimated irectly from the data and its value reported in addition to the nominal level. This is achieved through a combination of computer simulation and density estimation. The asymptotic validity of the procedure is proved for a number of common situations. A classical example is the nonparametric estimation of the variance of a population using the normal-theory interval. Here it is shown that the estimated coverage probability consistently estimates the true coverage probability ifthe population distribution possesses a finite sixth moment. The second approach is more traditional. Itis a procedure for modifying an interval to yield improved coverage properties. Given a confidence interval, its estimated coverage probability obtained in the first approach is used to alter the nominal level of the interval. The interval with this modified nominal evel is called a calibrated interval. In the case that the given interval is the normal-theory interval for the estimation of variance, the calibrated interval isproved to be asymptotically robust as long as sixth moments exist. As another application, the method is used to modify a bootstrap interval procedure for variance stimation. This leads to the derivation of a new bootstrap interval.